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Teacher cognition is seen as an important factor for the quality of instruction and, 
accordingly, student learning of a subject. However, in-depth research on these 
relations can only be done if a sound theoretical model for subject-specific teacher 
cognition (knowledge and competence/practical skills) and corresponding measures 
are available. This work aims at the development of reliable and valid measures for 
primary school teachers in mathematics. The subject-specific cognition is modeled as 
basic professional knowledge (BK) and further two components of reflective 
competence (RC) and action-related competence (AC). Using video-based items, we 
developed a computer-based standardized test and collected data of N = 85 primary 
mathematics teachers. We present results on the quality of the measures and have a 
closer look on the structure of teacher cognition. 


MEASURING TEACHER KNOWLEDGE 


Despite the broad consensus of the importance of teachers’ subject-specific cognition 
for the instructional quality of lessons and the learning of students, concepts to model 
these cognitive structures are still deficient. As a consequence, research is still narrow 
and focuses on single aspects of teacher cognition (Kunter, & Klusmann, 2010). For 
example, the facets of mathematics teachers professional knowledge has been 
elaborately described based on Shulman’s (1986) topology and measures for 
mathematics teachers’ content knowledge (CK) and pedagogical content knowledge 
(PCK) were developed successfully (e.g. COACTIV for secondary teachers in 
Germany, Baumert et al., 2010; MKT for elementary teachers in the US, Hill et al., 
2004; TEDS-M for pre-service teachers in various countries, Dohrmann, & Blomeke, 
2012). Some studies provided evidence that teacher knowledge is positively related to 
instructional quality and student learning (Baumert et al., 2010; Hill et al., 2007). 


These results strengthen the assumption that subject-specific professional knowledge 
of teachers guides their instructional actions (see also Hattie, 2009). It is consensus that 
teacher knowledge is highly specialized, context-bound, and needs to be flexible in 
order to cope successfully with emerging classroom situations. But so far there is little 
specific evidence how teachers use their subject-specific knowledge to create and 
conduct instruction. Moreover, the relationship between teacher knowledge and the 
ability to use this knowledge in and for action is considered as complex and divergent. 
Particularly, this divergence can be described by the two commonly known 
phenomena of inert knowledge and tacit knowledge: Knowledge can be inert, so that a 
teacher might perform a knowledge test well, but is not able to utilize this knowledge 
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in classroom situations. Contrary, if teachers hold tacit knowledge, they react well 
during classroom situations, but are not able to make this practical knowledge explicit 
in a knowledge test (Lindmeier, 2011). 


To address these issues, alternative approaches have been developed in order to 
investigate facets of teachers’ cognition “beyond” knowledge: Research is then 
focused on usable teacher knowledge (e.g. Kersting et al., 2012), on professional 
vision as knowledge-based process (e.g. Stiirmer et al., 2013; Vondrova, & Zalska, 
2013) or on constructs of competence (our approach, see below). All of these 
approaches account for the importance of the applicability of teacher knowledge and, 
hence, seek for valid methods to tests the applicability in standardized procedures. As 
video-vignettes are seen as a possibility to convey the typical demands of teaching, all 
these approaches make use of them. However, the ways of using the video-vignettes 
differ and are very specific for each approach (see Lindmeier, 2013). 


TEACHERS’ SUBJECT-SPECIFIC COMPETENCE 


Our approach accounts for the variety of cognitive demands of typical tasks of teaching 
and considers knowledge, skills, and abilities needed to master these demands. In a 
European tradition, we use the concept of competence to model these complex ability 
constructs (Koeppen et al., 2008). Therefore, we understand subject-specific teacher 
competences as learnable, highly context-specific, individual cognitive dispositions 
that are required to master the typical demands of teaching a subject, in our case 
mathematics. In general, there are two groups of typical teacher tasks that clearly differ 
in their cognitive demands: (1) Tasks that occur during preparation and 
post-processing of instruction and are coined by _ reflective demands, 
e.g. choosing an appropriate representation and/or correcting students work, and 
(2) tasks that are related to the instruction itself and characterized by the spontaneous, 
immediate, interactive, and concurrent demands of teaching. Accordingly, Lindmeier 
(2011) suggest a subject-specific model for teacher cognition with three components: 
A basic knowledge (BK) component comprising teacher knowledge (CK and PCK). 
Two complementary components of competences are conceptualized as reflective 
competences (RC), and action-related competences (AC). The RC holds abilities and 
skills required for pre- and post-instructional tasks, whereas the AC comprises the 
abilities needed to fulfill the demands during instruction. Thus, for both components, 
basic knowledge is seen as a prerequisite that has to be enacted in situations related to 
teaching mathematics. However, the way of using this knowledge differs, as aims, 
situations, and retrieval modes differ between out- and in-classroom teacher work. 


In a feasibility study with video-based measures, the components could be separated 
empirically for secondary in-service (N = 28) and prospective (N = 22) teachers 
(Lindmeier, 2011). A replication with a larger sample has to follow. In the present 
study, we followed the approach to model the subject-specific cognition of primary 
mathematics teachers. 
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Primary school teachers in Germany 


Primary school teachers in Germany teach up to grade 4 and are specialized in one or 
two subjects. Typically, the children are taught mainly by a class teacher, which 
implies that mathematic is also taught by teachers without formal education for this 
subject. For Germany, Richter et al. (2012) report that up to 48% of primary teachers 
teach mathematics as not-certified teachers in some federal states. Clearly, German 
pre-service teachers that are educated as “primary generalists” exhibit a significantly 
lower content-specific knowledge than pre-service teachers that are “primary 
mathematics specialists” (Senk et al., 2013). It is assumed that this lack of 
subject-specific knowledge affects the quality of instruction and, hence, student 
lear-ning. However, the findings are inconsistent for this assumption (for Germany: 
Richter et al., 2012; Tiedemann, & Billmann-Mahecha, 2007; for an international 
overview: Hattie, 2009). In addition, these findings rely usually on measures of teacher 
knowledge (at best, or even only on information about certification) and do not account 
for possible differences of teacher abilities to use this knowledge in and for instruction, 
as apt measures are still missing. 


Competence structure 


If teacher competences “beyond” knowledge can be measured, the relation between 
knowledge and competences could be investigated. This can be used to describe 
different profiles of teacher cognition, to identify favorable profiles, and thus lay 
ground for evidence-based specific professional development programs, e.g. for 
not-certified teachers. As mentioned above, we assume that the BK 1s a prerequisite for 
the two competence components. For the interaction between RC and AC two 
assumptions can be derived theoretically: On the one hand, considerations that account 
for the instruction-related knowledge as highly-specialized craft knowledge that is in 
addition strictly situated in practice (Leinhardt, & Greeno, 1986) may lead to the 
hypothesis that AC and RC do not show any relation beyond their common rooting in 
BK (“independent AC/RC hypothesis”). On the other hand, other views on the work of 
a teacher as reflective practitioner (Sch6n, 1990) would lead to the expectation that RC 
plays a mediating role between knowledge and AC (“mediating RC hypothesis’’). 
Lindmeier (2011) found indications for the independency of RC and AC for secondary 
teachers. 


RESEARCH QUESTIONS 


The aim of our study was to adapt the three-component model of teacher cognition for 
primary mathematics teachers and develop standardized measures for individual 
diagnosis of the components in order to allow for a structural investigation of teacher 
cognition. Therefore, we worked on the following research question: (1) Is it feasible 
to develop valid and reliable instruments to assess the subject-specific components of 
primary teachers as BK, RC, and AC? (2) Which relations will prevail between the 
assessed competences? In particular, how are the relations between BK, RC, and AC? 
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METHODS 
Sample 


Overall, N = 85 teachers participated in our studies. All participants of this 
convenience sample teach mathematics at primary schools in Schleswig-Holstein (a 
federal state of Germany). The age of the teachers ranges between 25 and 63 years (M 
= 45.3). The majority of the participants were female (89.4%) and nearly half of the 
sample (51.8%) was certified in mathematics. 


Instruments 


For adequate competence measures, items have to reflect appropriately the 
context-specific demands that were used to conceptualize the competence (Koeppen at 
al., 2008). Overall, we realized our measures as standardized, computer-based tests. 
We used the software vKID (Lindmeier, 2011), where a variety of item types can be 
realized. Specifically, we tailored the items to mirror the characterizing demands of the 
three components BK, RC, and AC. As mentioned before, items based on 
video-vignettes are considered as especially suited to convey the demands of teaching, 
as they account for the complexity of instructional situations. Moreover, recent studies 
illustrated that video-based items are indeed appropriate to elicit teacher abilities that 
go “beyond” teacher knowledge (see above). 


Hence, for the AC measure, we developed short video-vignettes of typical classroom 
situations from primary mathematics instruction. Since AC is especially characterized 
by its spontaneous and immediate demands, we used a direct oral answer format. 
Teachers had to answer under time pressure and their reactions were audio recorded. 
Due to the spontaneous oral answers, this item type differs from other video-based 
approaches and has been already successfully implemented by Lindmeier (2011). AC 
was measured with 8 items by two types of video-based item: Some items were 
focused on the teachers’ abilities to address “students’ cognition” and teachers had to 
cope with a student’s individual strategies and misconceptions. The other items 
focused on abilities to deal with “representations and explanations” in instruction. The 
RC (9 items) was captured by picture- or video-based items and the teachers’ answers 
were partly direct and audio recorded, partly written. According to the 
conceptualization of RC, the teachers were instructed to act as they were in a post- or 
pre-lesson period. They were allowed to have sufficient time for reflection. We used 
two different types of items within the RC measure: First, the item type “students’ 
cognition” focused on the evaluation of student work, so that e. g. the rationale of a 
systematical error had to be analyzed. Second, the item type “representation and 
explanation” focused on planning snippets of lessons that can be related to a given 
video-vignette. Finally, the BK (9 items) was assessed with items comparable to the 
items from other studies with focus on teacher knowledge (see above, Dohrmann, & 
Blomeke, 2012, Hill et al., 2004). However, we presented the items via computer, so 
that the teachers answered written items by typing their answer. The items were 
constructed to mirror content knowledge as well as pedagogical content knowledge. 


3 - 436 PME 2014 


Articles published in the Proceedings are copyrighted by the authors. 


Knievel, Lindmeier, Heinze 


However, we do not differentiate these in our analyses. We restricted the content for 
the whole test to arithmetical topics such as numbers, the place-value system, 
arithmetical strategies, and operations. The teachers completed the test individually at 
their own pace (they needed 42-88 min, M = 68 min to answer the test). In the 
beginning, a technical introduction with sample items was given to avoid technical 
issues with the handling of the computer. Furthermore, information on background 
variables was collected, including information on _ variables to assess 
criterion-irrelevant difficulties depending on the computer usage. 


All audiotaped answers were transcribed before two trained persons coded them 
independently with a detailed manual. The codes were based on findings from 
pri-mary mathematics education and accounted for aspects of high-quality instruction. 
The inter-rater reliability was moderate to high with a range of Cohen’s Kappa of 
k = .74-.94. The rate of missing data due to skipped items (time-on-task < 2 sec), 
skipped videos or technical problems was low (2.0%). We handled the missing 
information in background variables by case-wise deletion. Missing data in knowledge 
and competence measures is accounted for by using individual mean values of the 
answered items per scale (relative solution rates). Further, we used a cut-off value, so 
that scores were only computed if at least 75% of the data per scale was available. The 
missing data technique we used is discussed, as the resulting bias was not 
systematically assessed so far. However, if the rate of missing data is low and the mean 
item solution rates for items with missing data and without is comparable, it is seen as 
not problematic (Enders, 2010). 


RESULTS 
Feasibility: Reliability and Validity 


For evaluating the feasibility of measuring the three proposed scales, we assessed the 
reliability in terms of internal consistence (Cronbach’s Alpha) and compared it to the 
overall scale. The results of the analyses are reported in Table 1. 


Cronbach’s Alpha Est. Cronbach’s Alpha’ 
Scale (Precision) for scale length 11 Solution Rates (SD) 
BK (7 items) .68 (.03 ) ld -55(.12) 
RC (7 items) 64 (.04) 3 -49(.20) 
AC (8 items) .69 (.02) .76 .38 (.15) 
Overall scale 83 (01) 70 47.17) 


(22 items) 


'Spearman-Brown prediction formula. 


Table 1: Cronbach's Alphas and estimated Cronbach's Alphas for scale size 11 
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The BK and RC scale had to be reduced by two items each, due to obvious item misfit 
or/and extreme item difficulties. The reliabilities of the intended scales are sufficient 
considering the heterogeneous item types. Furthermore, the estimated Cronbach’s 
Alphas for the BK, RC, and AC scale show slightly higher internal consistencies than 
the overall scale. Thus, the three scales could be used as intended. 


In order to evaluate the criterion validity, the mathematics certified teachers are 
compared with the not-mathematics-certified teachers. Table 2 shows the results of a 
t-test between the groups. The mean differences for BK, RC, and the overall scale are 
of medium to high effect size. But for AC, no statistically significant difference 
between the groups was found. 


Certified Teachers Not-certified Teachers Effect Size 

Scale N=43 N=41l t(82) Cohen’s d 
BK 64.63 46.62 oe 87 
RC 55.58 41.68 2.02" .62 
AC 37.39 37.39 0.00 n.s. .00 
Overall scale 52.10 41.69 2.86* .63 


Table 2: T-Test for group differences between math-certified and not-math-certified 
teachers (* p < .01, ** p< .001) 


Competence Structure: Relations between the three components 


Correlations were used to assess the relations between BK, RC, and AC. We found BK 
significantly related to RC (1(84) = .53, p < .01) and AC (r(84) = .26, p < .05). 
Moreover, the relation between RC and AC is statistically significant (r(84) = .40, p < 
01). In order to evaluate in detail the relationship between RC and AC, partial 
correlation (controlling BK) was calculated. Controlling for BK, the partial correlation 
of RC and AC remains r = .32 (p < .01). Hence, we rejected the “independent RC/AC 
—hypothesis” and assessed the “mediating RC-hypothesis”. As mentioned above, 
significant correlations exist between BK and RC and between RC and AC. A 
hierarchical linear regression showed that the original effect (8 = .26, p < .05) of the 
BK on AC decreased (8 = .07 n.s.), when RC was entered as a control variable. Sobel’s 
test confirms that the decrease of the effect was significant (¢ = 2.71, 
p<.01). Hence, we find evidence for a mediator model, with RC mediating the relation 
between BK and AC. Using RC and BK as a predictor, 17% of the variance of the 
AC-scores could be explained. 


DISCUSSION 


The aim of this study was to develop competence measures for primary mathematics 
teachers’ cognition based on the proposed model. The empirical results of the 
reliability analyses indicate that the three components of subject-specific teacher 
cognition could be operationalized in distinct measures. The scales were seen as 
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coherent enough, despite the inhomogeneous item types, item formats, and content 
areas. Consequently, the results of Lindmeier (2011) could be transferred to the 
context of primary teachers. Further, the measures are examined with the aid of the 
different groups, namely teachers certified and not-certified in mathematics. The 
instrument is able to indicate the expected differences and a mean difference for BK, 
RC, and the overall competence is observed. Surprisingly, no difference could be 
observed for the scale of AC. One explanation could be the nature of the 
video-presented classroom situations: We decided to use very prototypical situations 
that are expected to appear in every primary mathematics classroom, so that teachers 
who regularly teach mathematics could have learned to deal with these situations. But 
this consideration would lead to a limiting point of our study in respect to the predictive 
validity of our measures for instructional quality: If a teacher reacts appropriately in a 
very typical situation that is clearly focused in our vignette, 1t remains unclear whether 
the teacher would also “see” this situation during lesson (see above, professional 
vision). Consequently, future research has to show how far the measures are valid for 
predicting classroom performance and student achievement. 


Our analyses of the relations between the measures showed that the relations between 
BK and AC and between AC and RC are of medium size, between BK and RC even 
large. Hence, the scales for RC und AC capture facets of teacher cognition that go 
“beyond” knowledge in the sense that the scales depend on professional knowledge but 
mirror further abilities to use knowledge in typical teaching tasks. Especially, for the 
AC that was measured by video-based items, these findings are in line with the studies 
that follow related ideas to measure the applicability of teacher knowledge. 


The relationship between RC and AC 1s of medium size even when controlling for BK. 
Consequently, in this study, we do find a stable relation between RC and AC for 
primary teachers, what differs from our findings for secondary teachers. Further 
analysis shows that RC mediates the relationship between BK and AC, but this 
mediation explains only a small part of the variance in AC. This can be interpreted as 
follows: Although AC depends on BK and RC, AC clearly differs from BK and RC. 
We would explain this difference with the underlying different cognitive demands. 
Altogether, our findings underpin the important role of teacher knowledge. But at the 
same time we can elicit teaching-related competences that go beyond knowledge and 
thus would conclude that basic knowledge might be not sufficient to describe the 
variance of subject-specific teacher cognition. 
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